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ABSTRACT 

This paper lists practices recommended by the Council 
for Exceptional Children's Division for Early Childhood concerning 
program evaluation in early intervention and early childhood special 
education programs for infants and young children with special needs 
and their families. An introductory section notes the guiding 
assumptions upon which the recommended program evaluation practices 
are based. A framework for the recommend3d practices is then 
discussed, categorizing the indicators according to four attributes 
of a program evaluation: utility, feasibility, propriety, and 
technical adequacy. Twenty-three recommended practices are then 
listed, within the categories of utility, feasibility, propriety, and 
technical adequacy. The recommended practice indicators are not 
exclusively bound to any particular evaluation model, and are not 
specific to a single program evaluation decision, evaluation object, 
or evaluation type. (Contains 10 references.) (JDD) 
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Recommended practices for Program Evaluauon are based on a number of guiding 
assumpUon, Pirst, program evaluaUon is no, a homogeneous construe, T,ere are broad 
spectra of potential purposes, methods and audience, for program evaluation in early 
intervention. likewise .he* is substantial diversity in me characterises of early 
intervention programs, children and families served by such programs, and the contorts of 
su ch program, T* recommended pracUces for program evaluaUon reflect fundamental 
principles *at can be broadly appUed within the fleld rather man speciflc pracUees tha, are 
recommended for speciflc purposes or contexts of evaluaUon. « is, me indicators 
sacrif.ee speciflcity for general* It is anticipate, mat the principies can be reflned 
ana operaUonalized to better serve more narrowly focused needs of speciflc evaluations. 

Second, it was assumed that the availability of estabiished standards for the pracUce 
of program evaluaUon in eUucaUon and related fleids (Join, Committee on Standards of 
Educauonal EvaluaUon, 1981; Rossi, 1982) provides an appropriate fonndaUon for 
developing standards appUcable to early intervenuon. Hre exisUng standards were 
Resize, and amended to represent the fleld. Again, the generality of the existing 
principles took precedence over specificity. 

U. final major assumption was that program evaluation (whether internal or external, 
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formal or informal) is conducted under limited resources, limited time, and administrative 
and political constraints. Due to such limitations, it is not reasonable to assume that all of 
the recommended practice principles can be met equally well within a given evaluation. 
Therefore, the "necessity 1 * of a principle should be viewed in the context of the decisions and 
constraints of the evaluation. 

Overview 

The recommended practices primarily represent a synthesis of program evaluation 
standards presented by the Evaluation Research Society (Rossi, 1982) and the Joint 
Committee on Standards for Educational Evaluation (1981). Stufflebeam (1990) states that a 
number of studies which have examined the two sets of standards have found them to be 
largely overlapping. The framework of the recommended practices in Program Evaluation 
follows the outline of the Standards for Evaluations of Educational Programs, Projects, and 
Materials (Joint Committee on Standards for Educational Evaluation, 1981). The framework 
categorizes the indicators according to four attributes of an evaluation: utility [those 
practices that support the ability of an evaluation to serve the needs of evaluation 
stakeholders (program administrators, program staff, parents, funding agents) in a manner 
that is credible, informative, timely and influential]; feasibility (those practices that support 
the conduct of program evaluation within the constraints imposed by limited resources, time 
demands and the political subtext of early intervention programs); propriety (the ethical and 
constitutional rights of participants in, and audiences of program evaluation and the 
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responsibilities of evaluators to protect such rights); and technical adequacy 1 (those practices 
that support the gathering, analysis, and interpretation of information in ways that are valid, 
reliable, accurate, representative, fair, and replicable. 

A number of evaluation models are available [e.g. CIPP (Stufflebeam, 1983), goal- 
free evaluation (Scriven, 1972), responsive evaluation (Stake, 1980); illuminative evaluation 
(Parlett, 1981), discrepancy evaluation (Provus, 1971)]. The recommended practice 
indicators are not exclusively bound to any particular model. Similarly, the indicators do not 
encourage specific methodologies for planning, designing or managing evaluations; gathering 
and analyzing information, or disseminating results. Rather, evaluations should employ the 
model and methods that best serve both the needs of the evaluation and the principles of 
recommended practice. This eclectic orientation to models and methods of program 
evaluation is supported by the framers of the existing standards. 

None of the recommended practice principles are specific to a single program 
evaluation decision (e.g., program revision), evaluation object (community needs, child 
performance, program cost, staff performance, materials, program components, parent 
participation, etc.) evaluation type (formal vs. informal, conducted by internal staff or 
external evaluator, large-scale vs. small). As with die original standards, it was hoped that 
the recommended practices could be applied by a range of practitioners. These include early 
interventionists interested in improving the quality of services they provide to children and 
families; administrators of local early intervention programs interested in evaluating the 



*This attribute is titled "accuracy" within the Joint Committee standards but is renamed 
here for the purpose of clarity. 
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performance of staff; state-level administrators wishing to determine the cost-effectiveness of 
services provided to infants with special needs and their families; or a professional evaluator 
hired to evaluate the differential effects of two early intervention models* 

While the specific principles are not ordered in any sequence or hierarchy within the 
four major attributes, the attributes have been listed in an order recommended by the Joint 
Committee (Stufflebeam, 1990). The order of the attributes (utility, feasibility, propriety, 
and technical adequacy) is based on the following logic: (a) before undertaking an evaluation 
there is a need to be certain that the findings from an evaluation will be useful (e.g., in 
providing important feedback or assisting in decision-making) (b) if the evaluation is 
expected to produce some useful information it must be determined if conducting the , 
evaluation would be feasible and efficient give v, available resources to obtain and report 
information in time for its use; (c) if the evaluation is determined to be feasible, possible 
obstacles to conducting the evaluation within the bounds of propriety must be considered; and 
finally (d) if it is determined that the evaluation can be conducted ethically, legally, and 
responsibly, the attention and efforts focus on the technical aspects of the evaluation. 
Stufflebeam (1990) and Worthen and Sanders (1987) present a table which summarizes the 
Joint Committee's consensus on the relationship of their standards to ten common tasks of 
program evaluation. While the standards of the Joint Committee and the recommended 
practice indicators discussed here are different, there is sufficient overlap to make such a 
table informative to the reader interested in clarifying to which tasks specific principles most 
clearly apply. 

In addition to the consensual validation of these recommended practice principles, 
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supplemental evidence of their validity is available. Stufflebeam (1990) reviewed studies that 
have addressed the validity of the Joint Committee's standards (including their applicability 
to specific situations and disciplines) and concluded that while they are not a panacea, their 
content is sound and their applicability is well established. An annotated bibliography 
supporting each of the Joint Committee's standards was developed by Wildemuth (1981). 
Finally, various published resources in early intervention contain discussion of principles and 
guidelines similar to those comprising the recommended practices. 
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DEC Recommended Practices 
Program Evaluation 

Program evaluation consists of collecting and reporting information to answer significant questions about 
aspects of programs. Examples of such program aspects include child or family functioning, staff 
performance, educational materials, transition, classroom environments, parent participation, curriculum, 
program expenditures and community programming needs. Answers to such evaluation questions might 
serve: a) to make decisions within a classroom; b) to formalize policy; c) to determine the viability of 
implementing a new program; d) to modify and improve program practices; e) to determine how funds 
should be allocated; f) to support the continuation, expansion, or discontinuation of a program; and g) to 
demonstrate accountability or cost-effectiveness. 

Many of the following best practice indicators are congruent with standards presented by the Evaluation 
Research Society (Rossi, 1982) and by the Joint Committee on Standards for Educational Evaluation (Joint 
Committee, 1981; Worthen & Sanders, 1987). As there is substantial overlap between the two sets of 
standards and recommendations of task force members, individual citations for the best practice indicators 
are not provided. The framework for grouping these indicators according to four attributes of evaluation 
(utility, feasibility, propriety, and technical adequacy) approximates the approach used by the Joint 
Committee (1981). 

Utility 

PEL Program evaluators or staff identify audiences involved in or affected by the evaluation so their 
needs and expectations can be addressed and their cooperation obtained. 

PE2. The evaluator(s) must be competent (i.e., received training) to perform the desired evaluation and 
must be trustworthy. 

PE3. Information collected is sufficient in scope, and derived from sources sufficient in credibility, to 
address pertinent evaluation questions. 

PE4. Program evaluators describe thoroughly their assumptions, perspectives, methods and rationale used 
to generate and interpret findings so that the audience can judge the basis for decisions. 

PE5. Evaluation reports clearly describe the purpose and rationale of the evaluation, the specific 
evaluation questions addressed, the program aspects being evaluated (materials, program 
components, staff performance, parent participation), the programmatic context, the evaluation 
procedures (design, data collection, analysis, etc.), findings, conclusions, and recommendations. 

PE6. Program evaluators present their findings clearly, completely, and fairly in language the audiences 
understand. 

PE7. Program evaluators present multiple findings or recommendations in order of relative importance. 
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PE8, Program evaluators disseminate their findings in a timely manner so that audiences can best use the 
information. 

Feasibility 

PE9, Program evaluators conduct evaluations with minimal disruptions to the program, staff and families. 

PE10, Before beginning data collection, program evaluators, adminstrators, and or staff determine that the 
evaluation plan is an effective, ethical, legal and fiscally responsible use of resources. 

Propriety 

PE11. Program evaluators report findings in a legal and ethical manner that is in due regard for the rights 
and welfare of participants and audiences. 

Technical Adequacy 

PE12. Program evaluators describe the focus of an evaluation (e.g., program, materials) as precisely as 
possible. 

PE13. Program evaluators describe and examine precisely the context in which the object of the evaluation 
exists to determine the influences of the context on the object. 

PE14. Program evaluators describe and monitor the purposes, designs, and procedures of an evaluation in 
enough depth and precision to permit adequate critique, management and/or replication of the 
evaluation. 

PE15. Whenever appropriate, program evaluators assess multiple sources of information. 

PE16. Program evaluators describe and justify sources of information and sampling procedures so that the 
adequacy and defensibility of information can be assessed. 

PE17. Measurement instruments and procedures are appropriate for the characteristics of the respondent 
(e.g. handicapping condition, gender, language, culture, developmental level). 

PE18. Program evaluators select, develop, and use measurement instruments and procedures in ways that 
assure that the interpretation of the information is reliable and valid for the intended use. 

PE19. Program evaluators pilot-test locally developed instruments/procedures to insure technical adequacy, 
validity and reliability. 

PE20. Program evaluators systematically review and correct (if necessary) the collection, storage, 
management, analysis and reporting of program evaluation data. 

PE21. Program evaluators use for data analysis the simplest systematic procedures that are appropriate, 
given the purposes and design of the evaluation and the nature of the data. 



PE22. Program evaluators describe and justify procedures for analyzing qualitative or quantitative 
information. 

PE23. In assessment reports, program evaluators distinguish between objective findings (e.g. statistical and 
practical interpretations of information), opinions, judgments, and recommendations. 
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